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Variational selective image segmentation models aim to extract a particular 
object in an image depending on a set of user-defined prior points. The current 
model suffers from high computational costs due to the traditional total 
variation function that results in a slow segmenting process. In addition, it is 
not designed to segment images with intensity inhomogeneities. In this 
research, we formulate a new variational selective image segmentation model 
based on the Gaussian function. A Gaussian function is proposed to replace 
the traditional total variation function to regularize the variational level set 
function. To segment images with intensity inhomogeneities, the local image 
fitting idea was incorporate into the formulation. The efficiency of the 
proposed model was then assessed by recording the computation time while 
the accuracy was measured using Jaccard and Dice similarity values. 
Numerical experiments using synthetic, natural, and medical images 


Variational level set demonstrate that the proposed model is about 6 times faster than the existing 
model, while the Jaccard and Dice values are about 11% and 7% higher, 
respectively, compared to the existing model. In the future, this research can 
be extended further into a 3-dimensional modeling and vector-valued image 


framework. 
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1. INTRODUCTION 

In order to examine and separate a digital image into different components that may be used for 
fundamental applications such as in the disciplines of medical diagnostics [1], and object recognition [2]—[5], 
image segmentation is required. Intensity inhomogeneity is one of the main obstacles to image segmentation, 
which is produced by errors in image acquisition, the effects of illumination, and other environmental factors. 
Researchers may make mistakes in interpreting an image as a result of the presence of intensity inhomogeneity 
in the image. As a result, segmenting an image with intensity inhomogeneity increasing the interwst of many 
researchers. Many approaches for image segmentation have recently been developed. The segmentation 
approaches can be classified into variational and non-variational segmentation approaches. Variational 
approaches use calculus of variations to minimize the cost energy function where the basic idea is to define an 
objective function and apply optimization procedures to obtain optimality (minimum or maximum), while non- 
variational methods are formulated based on a heuristic approach [6]. 
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The methods of the non-variational model include region growing [7] and thresholding [8]. For a 
region with low contrast, noisy images, and being near to a neighboring object, region-growing and 
thresholding methods may give unsatisfactory results as all features or objects (including the image noise) in 
an image may be segmented by these two methods. Other models also utilise the machine learning-based 
approach. The methods used include convolutional neural networks (CNN) [9] and U-Net [10]. Studies have 
proven that these machine learning-based approaches to non-variational image segmentation are effective for 
image segmentation [11]. However, the machine learning-based approach does have its disadvantages, such as 
being too dependent on data and that the process of segmenting images is unknown [12]. 

The diverse characteristics of a set of images have been shown to be effectively handled by the 
variational image segmentation techniques, which also provide high-quality processing capabilities for imaging 
[13]-[15]. These benefits led our research to concentrate on variational image segmentation methods. An image 
is viewed as a function in variational image processing, with its sampling corresponding to an image's discrete 
matrix form. Chan and Vese [16], Zhang et al. [17], and Jumaat and Chen [18] introduced some examples of 
effective variational image segmentation models in two-dimensional formulation, while [19] formulated the 
three-dimensional model. 

Variational image segmentation employs both global and selective segmentation techniques. The 
global segmentation approach refers to techniques for segmenting all objects boundaries in observed images. 
The famous model was developed by Chan and Vese [16]. The Chan and Vese model [16] is abbreviated as 
the CV model in this research. Let z = Zp (x, y) be an image in domain D. In the CV model, the assumption 
made is that Zp is formed by two (2) main regions, where the unknown contour y separates the regions. Inside 
y, assume the region D, represents the targeted object with the unknown intensity value b,. Outside y, the 
image intensity is approximated by the unknown value b, in Dy = D\D,. Then, with D = D, U Dg, the CV 
model minimizes as shown in (1). 


min {CV (y, by, ba) = w length (y)+A; J, (20 ~ bs)? dxdy + Aa fp, Zo — be)? dxdy } (1) 


Here, the unknown constants b, and bz are considered as the approximate piecewise constant intensities of the 
mean values of Zp) inside and outside the variable contour y. The parameters ,/4, and Az, which are non- 
negative parameters, represent the weights for the regularizing term and the fitting term, respectively. To 
evaluate in (1) in whole domain, the level set method was applied and the Heaviside function, H (p(x, y)) = 
0.5(1 + (2/m) arctan(@/e)) and the Dirac delta function, 6 (o(x,y)) = H'(¢ (x, y)) were introduced. The 
function @ is a level set function. The contour y is defined as the zero-level set function, i.e., y = 
{(x, y) € D|@(x, y) = 0}. The constant ¢ is used to avoid the values of H($(~,y)) and 5(p(x,y)) tends to 
be zero which may lead to the failure of object to be extracted if it is far from the initial contour. Thus, as in 
(1) is transformed into (2). 


june haa) = 

{u J, IWH(p)| dxdy + Ay J, (Zo — by)?H($) dxdy + Ay J, (Zo — bo)?(1-H()) dxdy} 2) 
Fixing b, and b, as constants in CV(@, b,, by) leads to in (3): 

15 (HV. (VH/|VPL) — 2x5 (H) Zo — bi)? + A25(H) Zo — bz)? = 0. (3) 


where V®¢ represents the gradient of level set function @. As shown in (3) is called the Euler Lagrange (EL) 
equation with Neumann boundary condition, which was solved using the gradient descent method. However, 
the CV model is only effective for an image with homogeneous intensity. For images with intensity 
inhomogeneity, this method may produce unsatisfactory results. In addition, the model is computationally 
complex due to the existence of the curvature term in (3) that comes from the total variation term in (2). 

Consequently, Zhang et al. [17] proposed a global segmentation model based on local image fitting 
(LIF) which is capable of segmenting an image with intensity inhomogeneity. In addition, Gaussian filtering 
is used to regularize the level set function that results in a cheap computational cost and faster convergence. 
The LIF formulation is defined as (4). 


ZF = c,H() + c,(1 — H(@)). (4) 
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where c, = mean(Zp € ({(x,y) € DIp < O}NW,)) and cy = mean(Z E ({(x,y) € DI < 0} NW,)). 
Here, W,(x,y) is a Gaussian window with standard deviation o and size 4k+/ by 4k+/, k € Z*. Then, the 
local image fitting energy function is defined as (5). 


min {E“F (o) ==, (Zo — 24")? aD} (5) 


During evolution of level set function, Gaussian filtering was used to regularize the function to make the level 
set function smooth. By calculus of variations, the EL equation for the function in (5) is defined as 
—[zZo — c,H(@) - c(1 —-H (¢))| (c, — C2)5() = 0. Using the gradient descent method, the gradient descent 
flow is defined as 0¢/dt = [Zo —c,H(¢) - Gx(t - H($))| (cr — C,)5(@), used to solve the EL equation. 
The LIF model is suitable for global segmentation where all objects in an image will be segmented. To segment 
a specific object in an image, selective segmentation is more suitable. Mandal et al. [20] and Wei et al. [21] 
also proposed models to improve the traditional segmentation model and Chan and Vese’s model [16] 
respectively. Rada and Chen [22] assert that global segmentation techniques cannot be used to isolate a single 
object in an image. In order to do this, adopting techniques for selective segmentation is a more suitable way 
to complete the work. 

Selective segmentation is a technique of segmenting a particular object in an image based on a set of 
user-defined prior points called markers. With this attribute, selective segmentation has high potential to be 
incorporated with other computational techniques such as in medical imaging [23], [24], biometric application 
[25] and text processing [26]. Examples of effective models include Rada and Chen [22] and Jumaat and Chen 
[20]. The selective segmentation model by Jumaat and Chen [18], namely the primal dual selective 
segmentation (PDSS), is more effective to implement compared to Rada and Chen’s model [22] as the model 
is convex and less sensitive to initialization. In the PDSS model, the marker set is introduced, and it is defined 
as A = {w; = (xj, 97) €D,1 <i < n,} with n, > 3 marker points that will be placed near the targeted object. 
The function P(x, y) = f (x — Xp)? + (y — yp)? is the Euclidean distance of each point (x,y) € D from its 


nearest point in the polygon, P made up of ere Yp) € P, constructed from the user input set, A. Then, by 
adjusting H(®@) > u € [0,1], the PDSS function is then defined as (6). 


1 
in {PDss(u,w) =1J, Vulg dD +f, rwdD +0 f, PawdD +f, (u-w)? dD} (6) 


m 
u,we[0,1 


Here, g(x,y) is an edge detector function, r = (b, — Z))* — (bz — Zp)” is the fitting term and w is a dual 


variable. While this model is effective for the selective segmentation model, it may give unsatisfactory results 
for an image with intensity inhomogeneity due to the similar fitting term as the CV model is used in the PDSS 
formulation. In addition, the existence of total variation term in the formulation may increase the computational 
cost and results in slow convergence. 

Therefore, in this research, we propose a new model by incorporating the ideas from the PDSS model 
[18] and Zhang ef al. [17] into a new optimization formulation. We expected to see improvements in terms of 
efficiency and segmentation accuracy by modifying the PDSS model. 


2. METHOD 

In this section, the proposed selective segmentation model is presented. Three main ideas are 
incorporated in the formulation of the energy minimization function: i) A distance function which is vital to 
capture a specific object; ii) A local image fitting to deal with an image with intensity inhomogeneity and iii) 
The Gaussian function to regularize level set function. We define a marker set A = {w; = (xj, y;) €D,1< 
i < n,} with n, => 3 marker points that will be placed near the boundary of a targeted object in a given image 
Zo(x,y). Then, the energy minimization function of the proposed model termed the Gaussian regularization 
selective segmentation (GRSS*) is defined as (7): 


min {GRSS*() = 5 J, (2 — Fil) + fo —H(@))) aD + J, Pa HCG) 4D} (7) 


with f, (x,y) = kg * [H(b)zZol/ke * H(p) and f2(%, y) = kg * [1 — H()zZo]/k, * [1 — H(@)]. The function 
k, is a Gaussian kernel with standard deviation o such that k, = e*+y*)/20" The standard deviation o can 
be viewed as a scale parameter that controls the region-scalability from a small neighbourhood to the whole 
image domain [27]. The function f; and f, can be interpreted as the average of image intensities in a Gaussian 
window inside and outside the contour respectively, defined to handle an image with intensity inhomogeneity. 
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The parameter @ will restrict the contour from evolving too far from the targeted object. In practice, a small 0 
is needed for a simple image, while a large @ is suitable for a low contrast image or an object that is close to 
the neighbouring area. To obtain a smooth contour, the Gaussian kernel is used to regularize the level set 
function after each iteration. By calculus of variation, the associated EL equation for (7) is defined as (8). 


—5(b){[20 — fH) — fa(1 — HD) — fe) — Pa} = 0 (8) 
As shown in (8) can be solve using the gradient descent method to obtain the following gradient descent flow: 
< = 5(b){[20 — AHH) — fe(1 — HH) — fe) — Pa} (9) 
In other word, the GRSS* model is minimized by solving as (9). 


2.1. A new variant of GRSS? model 
We also propose a new variant of the GRSS* model, termed GRSS°, defined as (10). 


min {GRSS*(p) = 0 J, Pa (p)dD + J, [(bs ~ 20)? — (bs ~ %)*] H()dD} (10) 


The GRSS? model uses the idea from the PDSS model. Here, the total variation term in the PDSS model is 
removed to reduce the complexity. To regularize the level set function, the Gaussian kernel function is adopted 
in the GRSS’ model. The EL equation for the GRSS? model is 5(@) [(Z — b,)? — (Zp — bz)? + OPq] = 0 with 
the following gradient descent flow as (11). 


28 = 8 (){(@o — bs) — (2 — ba)? + Pa] a) 


2.2. Steps of algorithm for the proposed selective segmentation models 

The following algorithms show the steps involved to implement the new proposed models; the GRSS* 
model and the GRSS® model. MATLAB R2021a software with the CPU processor of Intel® Core TM-i7- 
1065G7 CPU @ 1.30 GHz with 8 GB installed memory (RAM) is used in the implementation. We first 
demonstrate the Algorithm 1 which is the GRSS? model. There are four steps taken, implemented in MATLAB 
software. The iteration process stops when the solution reaches the tolerance, tol or reaches the maximum 
iterations, maxit. The following is a description of the Algorithm 1: 


Algorithm 1. Algorithm for GRSS* model 


1. Set the tolerance, tol, maximum iterations, maxit, parameters values of o and @ and define 
the marker set A. 

2% Compute and initialize the level set function @. 

3. For iteration=1 to maximum iterations, maxit or ||@"*?-—@"l/ll@"ll < tol do 


Evolve the level set function @ based on Equation 9. 
Regularize @ by convolving with k,. 

end for 

4. The output @ will be defined as the final solution. 


Next, we discuss the Algorithm 2 which is the GRSS° model. All steps in Algorithm 2 are identical to 
the Algorithm | except in Step 2 and Step 3. In Step 2, the level set function is defined as a signed distance 
function while in Step 3 the evolving level set function @ is based on (11). The Algorithm 2 can be described 
as: 


Algorithm 2. Algorithm for GRSS° model 


1. Set the tolerance, tol, maximum iterations, maxit, parameters values of o and @ and define 
the marker set A. 

7 Compute and initialize the level set function @ as a signed distance function. 

3. For iteration=1 to maximum iterations, maxit or ||@"*+—@"||/|l6”|| < tol do 


Evolve the level set function @ based on Equation 11. 
Regularize @ by convolving with k,. 

end for 

4. The output @ will be defined as the final solution. 


Indonesian J Elec Eng & Comp Sci, Vol. 29, No. 1, January 2023: 277-285 


Indonesian J Elec Eng & Comp Sci ISSN: 2502-4752 ) 281 


3. RESULTS AND DISCUSSION 

In this research, the performance of the proposed selective segmentation models, GRSS* and GRSS° 
were compared with the existing selective segmentation model, PDSS [18]. Two similarity coefficients namely 
Jaccard (JSC) and Dice (DSC) were computed. A perfect segmentation quality is indicated by a value of 1 
while a poor quality of segmentation is indicated by a value of 0. For all experiments, the tolerance, tol = 
107° and the maximum iterations, maxit=200. The values of o to regularize the level set function in Algorithms 
1 and 2 are in the range of o = [0.1,0.45] while the values of o to solve (9) of Algorithm 1 range from o = 
[5,33] of window size 4k+/ by 4k+/, k € Z*. The test images with their ground truth solutions are obtained 
from [28]-[30]. Figure 1 demonstrates the segmentation results of all models in segmenting natural images. 
The first and fifth columns show the natural test images with the marker set in green. The second and sixth 
columns demonstrate the results for GRSS* model. The results for GRSS° model are indicated in the third and 
seventh columns while the fourth and the last column show the results for the PDSS model. The values of 0 
range from 6 = [150,2000]. 


Test Image GRSS* GRSS? Test Image GRSS* GRSS? 


k 
la 
2a 


Figure 1. Segmentation results on natural images 


By visual observation, all models successfully segmented the targeted object. However, our GRSS* 
model gave more accurate results, especially in segmenting the big bird in Figure 1(2b) and the mushroom in 
Figure 1(4b). Table 1 shows the computation time, JSC and DSC values of each model for each natural image. 


Table 1. The computation time, JSC and DSC values for each model in segmenting natural images 


Test Image la 2a 3a 4a Sa 6a 
GRSS* 1.03/0.89/0.94 — 0.93/0.90/0.95_—_-1.87/0.92/0.96 —_7.38/0.94/0.97 —_2.13/0.93/0.96 _1.34/0.90/0.95 
Time/JSC/DSC 
GRSS° 1.68/0.90/0.95 — 3.96/0.87/0.93 —_0.61/0.92/0.96 —_1.10/0.87/0.93 _0.74/0.92/0.96 —_-0..73/0.88/0.94 
Time/JSC/DSC 
PDSS 37.83/0.89/0.94 — 30.43/0.87/0.93  7.14/0.92/0.96 — 6.72/0.86/0.92 — 10.23/0.92/0.96 —_10.39/0.88/0.94 
Time/JSC/DSC 


Besides qualitative analysis by visual observation, we also provide a quantitative analysis of the 
segmentation accuracy by evaluating JSC and DSC values as well as the computation time as indicated in 
Table 1. For all tested images, the computation times for our proposed GRSS? and GRSS? are faster than PDSS. 
This is due to the advantage of using Gaussian kernel to regularize the level set function which is more efficient 
compared to the total variation regularization term. For an image with high intensity inhomogeneity in test 
images 2a, 4a and 6a, the GRSS* model had higher JSC and DSC values compared to the other models. For 
test image 1a, the model GRSS? had slightly higher values of JSC and DSC than the GRSS? and PDSS models. 
Besides natural images, we also tested all models in segmenting medical images as demonstrated in Figure 2. 

The values of @ used range from 6 = [655,5000]. Figure 2(7a) is an x-ray hand image, Figure 2(8a) 
and (9a) are blood vessel images, Figure 2(10a-13a) are breast ultrasound images, while Figure 2(14a) is skin 
abnormality image. The qualitative analysis (by visual evaluation) indicates that the GRSS* model gave more 
accurate results. For quantitative analysis, we tabulated the computing time, JSC and DSC values for all models 
in segmenting the medical images in Table 2. 
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The data in Table 2 confirms that the GRSS* model is more accurate compared to GRSS° and PDSS 
as indicated by the JSC and DSC values except for test image 14a where the segmentation accuracy for all 
models are similar. In terms of computation time, the GRSS* model is faster than the other two models for test 
images 7a and 11a while for other test images, the GRSS° model is faster than GRSS* and PDSS models. To 
complete the experiment, we demonstrate the segmentation of all models in segmenting synthetic images as 
shown in Figure 3. The values of the 6 used range from@ = [15,1300]. 


Test Image 


Figure 2. Segmentation results on medical images 


Table 2. The computation time, JSC and DSC values for each model in segmenting medical image 


Test Image Ta 8a 9a 10a lla 12a 13a 14a 
GRSS* 0.62/0.81/ — 9.39/0.90/ 8.19/0.90/ 2.20/0.94/ 0.45/0.91/ 1.35/0.90 1.13/0.92/ 3.63/0.8 
Time/JSC/DS 0.90 0.95 0.95 0.97 0.95 / 0.96 0/0.89 
Cc 0.95 
GRSS? 3.02/0.79/ — 1.76/0.77/ —-1.99/0.72/0._—_-0.73/0.89/0. 0.60/0.85/ 0.63/0.84 0.82/0.80/ 1.00/0.8 
Time/JSC/DS 0.88 0.87 84 94 0.92 / 0.89 0/0.89 
Cc 0.91 
PDSS 54.16/0.7 = 12.42/0.7 17.76/0.7/ 8.46/0.88/ 9.85/0.85/ 16.85/0.8 — 15.09/0.79/0 —_9.63/0.8 
Time/JSC/DS 3/ 8/ 0.82 0.94 0.92 5/0.92 .88 0/0.89 
C 0.84 0.88 


GRSS? PDSS Test Image GRSS* GRSS? PDSS 


| ee 2. ee 2 
= = - = 
19a 19b 19¢ 19d 
20a 20b 20c 20d 
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Figure 3. Segmentation results on synthetic images 
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By visual inspection, the GRSS* model is more accurate compared to GRSS° and PDSS. Thanks to 
the local fitting term that gives advantage to the proposed GRSS* model in segmenting the synthetic images 
with intensity inhomogeneity, for test images 15a-19a the GRSS° and PDSS models struggled to segment the 
inhomogeneous intensity image. This shows that the average intensity fitting term in their formulation is 
insufficient to segment this kind of image. To further investigate, the data on the computing time, JSC and 
DSC values are recorded and tabulated in Table 3. 


Table 3. The computation time, JSC and DSC values for each model in segmenting synthetic images 


Test Image 15a 16a 17a 18a 19a 20a 21a 22a 
GRSS* 0.86/0.95/ 2.51/0.84 1.69/0.88/0. — 2.95/0.94/0. —_-6.98/0.87/0. —-0.61/1.0/ 0.59/1.0/ 0.74/1. 
Time/JSC/D 0.97 /0.92 93 97 93 1.0 1.0 0/1.0 
SC 
GRSS? 1.45/0.37/ 1.05/0.73/ — 0.69/0.84/0. —_-0.76/0.79/0.—:0.72/0.85/0._—_-0.69/1.0/ 0.74/1.0/ 0.79/1. 
Time/JSC/D 0.54 0.84 8 92 1.0 1.0 0/1.0 
SC 
PDSS 11.99/0.37/0 = _7.75/0.72/0 —-11.97/0.73/0 ——-11.95/0.72/ 10.63/0.84/  6.60/1.0/ —6.52/0.995/0. —-9.79/1. 
Time/JSC/D 54 84 84 0.84 0.91 1.0 998 0/ 
SC 1.0 


Based on Table 3, it is confirmed that the GRSS* model is more accurate compared to GRSS? and 
PDSS in segmenting synthetic images with intensity inhomogeneity which are images 15a-19a. For 
homogeneous synthetic test images (images 20a-22a), a perfect score of JSC and DSC values were obtained 
by all models except for the the PDSS model in segmenting test image 21a. For test images 15a, 20a-22a, the 
computation time for the GRSS* model was faster than GRSS° and PDSS while the GRSS° model was faster 
than other models in segmenting the remaining synthetic images. The slowest computing time was delivered 
by the PDSS model. Again, this is the evidence of the advantage of using Gaussian function to regularize the 
level set function in our GRSS* and GRSS? models. 

Based on Table 1, 2 and 3, for all 22 test images used, the average of computing times for GRSS?*, 
GRSS? and PDSS model are 2.66s, 1.19s and 14.73s respectively. The average values of JSC for GRSS?, 
GRSS° and PDSS model are 0.91, 0.84 and 0.82 respectively while the average values of DSC for GRSS?, 
GRSS? and PDSS model are 0.95, 0.90 and 0.89 respectively. These data demonstrate that GRSS? is about 2 
times faster than GRSS? and 12 times faster than PDSS model, while GRSS* model is 6 times faster than PDSS 
model. The highest values of JSC and DSC are given by GRSS? model which are about 9% and 5% higher than 
GRSS? respectively, while comparison between GRSS? model with PDSS shows that the JSC and DSC values 
for GRSS* are about 11% and 7% higher than PDSS respectively. 


4. CONCLUSION 

The selective segmentation method was utilized in this study to selectively segment a certain object 
in grayscale digital images. This study has presented two new models, termed GRSS* and GRSS°. The 
corresponding Euler Lagrange equations for each model were provided to minimize the functions. The 
equations were then solved in the MATLAB platform using the gradient descent algorithm. By analyzing the 
execution time and accuracy of segmentation results quantified in terms of JSC and DSC for each test image, 
the performance of each model was assessed both visually and numerically. The existing model, PDSS, on 
average, was slower than the GRSS* and GRSS? models. On the contrary, in average GRSS° performed faster 
than GRSS? model. Improvement in computing time was achieved in GRSS* and GRSS° models due to the 
application of Gaussian kernel to regularize the level set function instead of the famous total variation term. 
By visual observation, all models were able to segment a targeted object in each test image. High segmentation 
accuracy was delivered by the GRSS* model especially for images with intensity inhomogeneity. This 
observation was proven numerically by the values of JSC and DSC where the GRSS* model scored higher 
accuracy compared to the GRSS> and PDSS models. This achievement was due to the advantage of 
incorporating a local image information that fits in the GRSS* model that can help in segmenting an image 
with intensity inhomogeneity. In conclusion, the GRSS* model is more recommended due to its ability to 
handle image with intensity inhomogeneity. In future, the GRSS* model will be extended to colour images and 
3-dimensional images since the colour and 3-dimensional images have more information. 
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